Enhanced color palette systems and methods for infrared imaging

ABSTRACT

Imaging systems and methods are disclosed for generating enhanced visual representations of captured data such as infrared image data. For example, the perceived color distance or contrast between colors representing adjacent output levels (e.g., temperature or infrared intensity levels) are enhanced in visual representations of infrared images. According to embodiments, infrared image data values representing a scene may be mapped according to a color palette implemented using complementary colors as adjacent (e.g., successive) base colors or a sequence of colors, that repeats a predetermine set of hues with varying saturation and/or intensity, thereby increasing the color contrast between pixels representing subtle temperature differences in the scene. The color palette can be enlarged by mapping a larger number of distinct output levels to a larger sequence of colors, for example by increasing the bit-depth of the color palette, such that color transitions look smoother and more natural.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/US2016/030308 filed Apr. 29, 2016 and entitled “ENHANCED COLORPALETTE SYSTEMS AND METHODS FOR INFRARED IMAGING,” which is incorporatedherein by reference in its entirety.

International Patent Application No. PCT/US2016/030308 filed Apr. 29,2016 claims priority to and the benefit of U.S. Provisional PatentApplication No. 62/156,163, filed May 1, 2015 and entitled “ENHANCEDCOLOR PALETTE SYSTEMS AND METHODS FOR INFRARED IMAGING,” which is herebyincorporated by reference in its entirety.

International Patent Application No. PCT/US 2016/030308 filed Apr. 29,2016 claims priority to and the benefit of U.S. Provisional PatentApplication No. 62/156,166, filed May 1, 2015 and entitled “ENHANCEDCOLOR PALETTE SYSTEMS AND METHODS FOR INFRARED IMAGING,” which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

The present application generally relates to systems and methods fordisplaying captured infrared data values representing a captured sceneon a display, and, more specifically, for providing enhanced display andinterpretation of the infrared data values in false colors orpseudo-colors.

BACKGROUND

Thermal infrared images of a scene are often useful for monitoring,inspection, and maintenance purposes. Often a thermal imaging device isprovided to capture infrared information or data values indicative ofthe intensity of infrared energy received from the scene, and create orgenerate a visual representation of the captured infrared information.For example, infrared information may be presented in the form of aninfrared image, which represents infrared radiation emitted from anobserved real world scene.

Infrared radiation is not visible to the human eye; there are no naturalrelations between the captured infrared image data values of each pixelin an infrared image and colors of a visual representation of theinfrared image generated on a display. Therefore, informationvisualization processes, often referred to as false coloring orpseudo-coloring, are typically used to map captured infrared image datavalue of each pixel in an infrared image to a corresponding color orgrayscale displayed on a display according to a palette or look-up table(LUT).

Color used to depict thermal images is useful because it stimulates aperceived contrast response in the brain that exceeds the perceivedcontrast between gray levels. Traditional color palettes or LUTs aredesigned around colors of the rainbow, the colors of incandescentobjects at different temperatures, or other aesthetically appealingcolor arrangements, and typically map infrared image data values to alimited number of output color levels (e.g., 8-bit, or 256, colorlevels).

The perceived color contrast in the aforementioned examples is lowbetween adjacent 8-bit output color levels resulting in difficultydistinguishing consecutive temperatures of the captured image. As aresult, there is a need for improved techniques for visualization oflocal areas of interest in a displayed infrared image, particularly inregards to infrared imaging color palettes.

SUMMARY

Various techniques are disclosed herein for systems and methods, inaccordance with one or more embodiments, using at least one infrared(IR) imaging device (e.g., thermal IR imaging device) to provide anenhanced visual representation of captured IR (e.g., thermal) image dataof a scene via improved color palettes. For example, in someembodiments, color palettes utilizing complementary colors (andnear-complementary colors in some embodiments) in a color space (e.g.,an RGB color space) as adjacent (e.g., successive) base colors orbreakpoints may be used to depict temperature value transitions in aninfrared image representing a real world scene. Such palettes aredesigned using a scientific approach based on color theory andmathematical representations to maximize perceived contrast, notstrictly visual appeal.

According to the different embodiments described herein, the generatingand displaying of enhanced visual representations of IR images may beprovided, in particular, with regard to complementary colors or othercolor arrangements that increase the perceptual color distance (e.g.,the perceived color contrast) between pixels representing IR intensitylevels (e.g., temperatures) that are different but nearby each other.Therefore, in some embodiments, an easily interpretable visualization ofan IR image that may be chosen by the user is provided.

In some embodiments of the present disclosure, a method and a system areprovided for generating an improved visual representation of IR datavalues captured by an IR imaging sensor, via advantageous arrangementsof complementary colors in a palette or LUT that improve the perceivedcolor distance or contrast between nearby IR data values. For example,the method may include: receiving IR image data captured by an IRimaging sensor comprising a plurality of detector elements, wherein theIR image data comprises, for each detector element, a correspondingpixel having an IR data value representing an intensity of IR radiationreceived by the detector element; and generating a visual representationof at least a portion of the IR image data, wherein the visualrepresentation comprises, for each pixel of the at least a portion ofthe IR image data, a corresponding color-representing componentaccording to a color model, and wherein the generating the visualrepresentation comprises, for each pixel: determining an output levelbased on the IR data value of the pixel, determining a color value thatcorresponds to the determined output level according to a palette,wherein the palette comprises a sequence of color values correspondingto a range of discrete output levels, the sequence of color valuescomprising a series of base color values positioned at substantiallysimilar intervals partitioning the sequence, wherein color values in thesequence other than the base color values are interpolated betweensuccessive base color values, and wherein the series of base colorvalues comprises at least one pair of successive base color valuesrepresenting a pair of colors that are complementary to each other, andassigning the determined color value to the correspondingcolor-representing component of the visual representation.

In another example, the system may include: an IR imaging sensorcomprising a plurality of detector elements, the IR imaging sensor beingconfigured to capture IR image data comprising, for each detectorelement, a corresponding pixel having an IR data value that representsan intensity of IR radiation received by the detector element; a memoryconfigured to store a palette comprising a sequence of color valuescorresponding to a range of discrete output levels, wherein the sequenceof color values comprises a series of base color values positioned atsubstantially similar intervals partitioning the sequence, wherein colorvalues in the sequence other than the base color values are interpolatedbetween successive base color values, and wherein the series of basecolor values comprises at least one pair of successive base color valuesrepresenting a pair of colors that are complementary to each other; anda processor communicatively coupled to the IR imaging sensor and thememory, the processor being configured to generate a visualrepresentation of at least a portion of the IR image data, wherein thevisual representation comprises, for each pixel of the at least aportion of the IR image data, a corresponding color-representingcomponent according to a color model, and wherein the processor isconfigured to generate the visual representation by: determining, foreach pixel of at least a portion of the IR image data, an output levelbased on the IR data value of the pixel, determining a color value thatcorresponds to the determined output level according to the palette, andassigning the determined color value to the correspondingcolor-representing component of the visual representation.

In other embodiments of the present disclosure, a method and a systemare provided for generating an improved visual representation of IR datavalues captured by an IR imaging sensor, via a palette or LUT arrangedto utilize a large number of colors to improve the perceived colordistance or contrast between nearby IR data values. For example, themethod may include: receiving IR image data captured by an IR imagingsensor comprising a plurality of detector elements, wherein the IR imagedata comprises, for each detector element, a corresponding pixel havingan IR data value representing an intensity of IR radiation received bythe detector element; and generating a visual representation of at leasta portion of the IR image data, wherein the visual representationcomprises, for each pixel of the at least a portion of the IR imagedata, a corresponding color-representing component according to a colormodel, and wherein the generating the visual representation comprises,for each pixel: determining an output level based on the IR data valueof the pixel, determining a color value that corresponds to thedetermined output level according to a palette, wherein the palettecomprises a sequence of color values corresponding to a range ofdiscrete output levels, the sequence of color values comprising a seriesof base color values positioned at substantially similar intervalspartitioning the sequence, wherein color values in the sequence otherthan the base color values are interpolated between successive basecolor values, and wherein the series of base color values comprises twoor more subseries each representing a different saturation and/orintensity of a same set of predetermined hues, and assigning thedetermined color value to the corresponding color-representing componentof the visual representation.

In another example, the system may include: an IR imaging sensorcomprising a plurality of detector elements, the IR imaging sensor beingconfigured to capture IR image data comprising, for each detectorelement, a corresponding pixel having an IR data value that representsan intensity of IR radiation received by the detector element; a memoryconfigured to store a palette comprising a sequence of color valuescorresponding to a range of discrete output levels, wherein the sequenceof color values comprises a series of base color values positioned atsubstantially similar intervals partitioning the predetermined sequence,wherein color values in the sequence other than the base color valuesare interpolated between successive base color values, and wherein theseries of base color values comprises two or more subseries eachrepresenting a different saturation and/or intensity of a same set ofpredetermined hues; and a processor communicatively coupled to the IRimaging sensor and the memory, the processor being configured togenerate a visual representation of at least a portion of the IR imagedata, wherein the visual representation comprises, for each pixel of theat least a portion of the IR image data, a correspondingcolor-representing component according to a color model, and wherein theprocessor is configured to generate the visual representation by:determining, for each pixel of at least a portion of the IR image data,an output level based on the IR data value of the pixel, determining acolor value that corresponds to the determined output level according tothe palette, and assigning the determined color value to thecorresponding color-representing component of the visual representation.In other example systems and methods, the sequence of color values inthe palette of the method may comprise at least 512 color valuescorresponding to at least 512 discrete output levels.

The scope of the invention is defined by the claims, which areincorporated into this section by reference. A more completeunderstanding of embodiments of the invention will be afforded to thoseskilled in the art, as well as a realization of additional advantagesthereof, by a consideration of the following detailed description of oneor more embodiments. Reference will be made to the appended sheets ofdrawings that will first be described briefly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of an imaging system in accordancewith one or more embodiments of the present disclosure.

FIGS. 2A-2C illustrate example visual representations of an infraredimage generated according to prior art.

FIGS. 3A-3C illustrate example plots of the color values in the priorart palettes used to generate the prior art visual representationexamples of FIGS. 2A-2C.

FIG. 4 illustrates a palette in accordance with an embodiment of thepresent disclosure.

FIG. 5 illustrates how corresponding color values are determined for IRimage data values by looking up a palette, in accordance with anembodiment of the present disclosure.

FIGS. 6A-6B illustrate example visual representations of an infraredimage generated in accordance with embodiments of the presentdisclosure.

FIGS. 7A-7B illustrate example plots of the color values in the palettesused to generate the visual representation examples of FIGS. 6A-6B, inaccordance with embodiments of the present disclosure.

FIGS. 8A-8B illustrate close-up views of a portion of the example visualrepresentation of FIG. 6A, in accordance with embodiments of the presentdisclosure.

FIG. 9 illustrates an example plot of the color values in the palette ofFIG. 4, in accordance with an embodiment of the disclosure.

FIG. 10 illustrates an example visual representation of an infraredimage generated using the palette of FIG. 4, in accordance withembodiments of the present disclosure.

FIG. 11 illustrates a palette as a color wheel, in accordance with anembodiment of the present disclosure.

FIG. 12 illustrates an example plot of the color values in the paletteof FIG. 11, in accordance with an embodiment of the disclosure.

FIG. 13 illustrates an example visual representation of an infraredimage generated using the palette of FIG. 11, in accordance withembodiments of the present disclosure.

FIG. 14 illustrates a palette in accordance with an embodiment of thepresent disclosure.

FIG. 15 illustrates an example plot of the color values in the paletteof FIG. 14, in accordance with an embodiment of the disclosure.

FIG. 16 illustrates an example visual representation of an infraredimage generated using the palette of FIG. 14, in accordance withembodiments of the present disclosure.

FIG. 17 illustrates a flowchart of a computer-implemented method forgenerating a visual representation of an IR image, in accordance with anembodiment of the present disclosure.

Embodiments of the invention and their advantages are best understood byreferring to the detailed description that follows. It should beappreciated that like reference numerals are used to identify likeelements illustrated in one or more of the figures.

DETAILED DESCRIPTION

Systems and methods are disclosed herein to provide, according tovarious embodiments, enhanced visual representations and displays ofinfrared data values. For example, an infrared camera may be used tocapture thermal images. The thermal images may be processed byperforming a mapping process using improved palettes (or LUTs) accordingto various embodiments of the disclosure to create and display visualrepresentations advantageously having better distinguishable temperaturerange variations.

Hereinafter, the terms “visual representation of infrared data values,”“visual representation,” and “infrared image” are used interchangeably.

Method embodiments herein are typically described for a single frame ofinfrared data values for easy understanding; however, the methods areapplicable to any number of captured frames of infrared data values andmay be used in any thermal imaging device generating still images and/orvideo image sequences.

FIG. 1 shows a block diagram of one or more embodiments of infraredimaging system 100, e.g., an infrared camera, a tablet computer, alaptop, PDA, mobile device, desktop computer, or any other electronicdevice capable of being used in thermography for capturing thermalimages and/or processing thermal data. System 100 may represent any typeof infrared camera that employs infrared detectors.

Infrared imaging system 100 comprises, in one implementation, an imagecapture component 130 (with infrared sensors and possible inclusion ofnon-thermal imaging sensors such as near-infrared (NIR)/visible light(VL) imaging sensors), processing component 110, control component 150,memory component 120, display component 140. Optionally, system 100 mayinclude a sensing component 160 such as a motion sensor, light sensor,rangefinder, proximity sensor, moisture sensor, temperature sensor, orother sensing component as desired for particular applications of system100. System 100 may represent, for example, an infrared imaging device,such as an infrared camera, to capture and process images, such as videoimages of scene 170. System 100 may comprise an external unit, such as aportable device and/or a remote device that the processing component 110communicates with unidirectionally or bidirectionally.

In various embodiments of the present disclosure, processing component110 may comprise any type of a processor or logic device, such as aprogrammable logic device (PLD) configured to perform processingfunctions. Processing component 110 may be adapted to interface andcommunicate with components 130, 150, 120, 140, and/or 160 to performmethods and processing steps and/or operations, such as controlling aswell as other conventional system processing functions as would beunderstood by someone skilled in the art.

In one or more embodiments, processing component 110 may be a processorsuch as a general or specific purpose processor/processing unit (e.g., amicroprocessor), microcontroller or other control logic that comprisessections of code or code portions, stored on a computer readable storagemedium (such as memory component 120), that are fixed to perform certaintasks but also other alterable sections of code (stored on a computerreadable storage medium) that can be altered during use. Such alterablesections of code can comprise parameters that are to be used as inputfor the various tasks, such as the calibration of system 100, adaptionof the sample rate or the filter for the spatial filtering of theimages, or any other parameter related operations known to a personskilled in the art and applied without inventive skill.

In one or more embodiments, processing component 110 is communicativelycoupled and communicates with memory 120 where parameters are kept readyfor use by the processing component 110 and the images being processedby the processing component 110 can be stored if the user desires. Theone or more memory components 120 may comprise a selection of a RAM,disk drive, a floppy disk drive, a magnetic tape drive, an optical diskdrive, a CD or DVD drive (R or RW), USB, or other removable or fixedmedia drive.

In an embodiment of the present disclosure, memory component 120comprises one or more memory devices adapted to store data andinformation, including, for example, infrared and/or visible data andinformation. Memory component 120 may include one or more various typesof memory devices including volatile and non-volatile memory devices,including computer-readable medium (portable or fixed). Processingcomponent 110 may be adapted to execute software stored in memorycomponent 120 so as to perform method and process steps and/oroperations described herein. According to various embodiments of thedisclosure, memory component 120 may be configured to store one or morepalettes 112A-112N, which may be arranged as LUTs, functional modules,or other data structure or module implementations for example. The oneor more palettes stored in memory component 120 may be any one of thepalettes discussed herein in connection with various embodiments.

In one embodiment, one or more image capture components include one ormore infrared imaging sensors, e.g., any type of infrared detectorhaving a plurality of detector elements, such as for example a focalplane array (FPA) of microbolometers or other infrared detectorelements, for capturing infrared image data, such as still image dataand/or video stream data, representative of an image of IR radiation(e.g., temperature) received from a scene or object (e.g., scene 170).In one implementation, the image capture component 130 may comprise aninfrared imaging sensor that may provide for representing (e.g.,converting) the captured image data as digital data (e.g., via ananalog-to-digital converter included as part of the infrared sensor orseparate from the infrared sensor as part of the system 100). In oneembodiment, image capture component 130 may include a radiometricinfrared camera calibrated to map image data captured by the camera tocorresponding temperatures of objects in the image data.

In one or more embodiments, image capture component 130 may furtherrepresent or include a lens, a shutter, and/or other associatedcomponents along with the vacuum package assembly for capturing infraredimage data. Image capture component 130 may further include temperaturesensors (temperature sensors may also be distributed within system 100)to provide temperature information to processing component 110 as tooperating temperature of image capture component 130.

Image capture component 130 may include one or more additional imagingsensors such as visible light image sensor (e.g., charged-coupled devicesensor and/or a complementary metal oxide semiconductor sensor), ashort-wave (SWIR) infrared sensor, a mid-wave infrared (MWIR) sensor,and/or a low-light visible and/or near infrared (VIS/NIR) sensor such asan image intensifier. For non-thermal sensors, electron multiplying CCD(EMCCD) sensors, scientific CMOS (sCMOS) sensors, intensifiedcharge-coupled device (ICCD) sensors, and CCD-based and CMOS-basedsensors, as well as any other suitable non-thermal sensor to detect NIR,SWIR, and other non-thermal light may be included in image capturecomponent 130. Images obtained by imaging component 130 may be combinedthrough known processes by processing component 110.

In one or more embodiments of the present disclosure, system 100 may beconfigured to have two physically separate devices as part of imagingcomponent 130, such as a first device comprising an infrared imagingdevice 180 and a second device comprising a visible light imaging device190. Devices 180 and 190 may communicate with processing component 110and have an integrated memory component 120 or one physically separatefrom the two devices. In one aspect, processing component 110 may beadapted to process the infrared image data (e.g., to provide processedimage data), store the infrared image data in memory component 120,and/or retrieve stored infrared image data from memory component 120.For example, processing component 110 may be adapted to process infraredimage data stored in memory component 120 to provide processed imagedata and information (e.g., captured and/or processed infrared imagedata).

Processing component 110 may be adapted to perform a false color orpseudo-color operation on infrared image data and display processed dataon display component 140, thus, providing distinguished temperaturegradients of captured thermal values.

According to an embodiment, control component 150 may be configured toreceive input from a user, thus, enabling a user to provide input to theinfrared imaging system 100. According to an embodiment, the controlcomponent 150 comprises a selection of one or more control devices forinputting commands and/or control signals, such as an interactivedisplay (e.g., a touch or pressure sensitive display, a joystick, amouse, a keyboard and/or record/push-buttons).

In another embodiment, system 100 may be handheld and/or portable, aswell as communicate bidirectionally with a remote device (e.g.,computer, cellular phone, electronic tablet, or any other electronicdevice).

In one or more embodiments, processing component 110 may communicatewith a remote device. The communications interface may comprise aselection of serial wired communication, Local Area Network (LAN),Metropolitan Area Network (MAN), Global System for Mobile Network (GSM),Enhanced Data GSM Environment (EDGE), High Speed Downlink Packet Access(HSDPA), Wideband Code Division Multiple Access (W-CDMA), Code DivisionMultiple Access (CDMA), Time Division Multiple Access (TDMA),Bluetooth®, Zigbee®, Wi-Fi, Voice over Internet Protocol (VoIP), LTEAdvanced, IEEE802.16m, WirelessMAN-Advanced, Evolved High-Speed PacketAccess (HSPA+), 3GPP Long Term Evolution (LTE), Mobile WiMAX (IEEE802.16e), Ultra Mobile Broadband (UMB) (formerly Evolution-DataOptimized (EV-DO) Rev. C), Fast Low-latency Access with Seamless HandoffOrthogonal Frequency Division Multiplexing (Flash-OFDM), High CapacitySpatial Division Multiple Access (iBurst®) and Mobile Broadband WirelessAccess (MBWA) (IEEE 802.20) systems, High Performance Radio MetropolitanArea Network (HIPERMAN), Beam-Division Multiple Access (BDMA), WorldInteroperability for Microwave Access (Wi-MAX), infrared communicationand ultrasonic communication, etc., but is not limited thereto.

In one or more embodiments, system 100 is configured to capture infraredimage data values, which represent infrared radiation emitted,transmitted, and/or reflected from an observed real world scene.Additionally, system 100 may correct or calibrate the captured datavalues via applying pre-determined infrared temperature calibration dataparameters to map and scale the captured data values for display as aninfrared or thermal image singly or combined with a visual light image(e.g., overlaid or fused).

In some embodiments, a machine-readable medium stores non-transitoryinformation comprising a plurality of machine-readable instructions. Oneor more processing components of the system execute such instructionsand cause the system to perform a method according to variousembodiments of the disclosure. For example, such instructions, whenexecuted by processing component 110, may cause system 100 to perform amethod for generating an improved visual representation of IR datavalues captured by IR imaging device 180, via advantageous colorarrangements/mappings in one or more palettes or LUTs 112A-112N storedin memory component 120 to improve the perceived color distance orcontrast between nearby IR data values, according to various embodimentsof the disclosure. In some embodiments, processing component 110 may beconfigured (e.g., by hardwired circuitry, software, firmware, or anycombination thereof) to perform such a method according to variousembodiments of the disclosure.

In one or more embodiments, processing unit 110 may be afield-programmable gate array (FPGA) or other types of programmablelogic device (PLD), which may be configured using a hardware descriptionlanguage (HDL) to perform a method according to various embodiments ofthe disclosure.

The terms “computer program product”, “computer-readable medium”, and“machine-readable medium” may be used generally to refer to media suchas memory component 120 or the storage medium of processing unit 110 oran external storage medium. These and other forms of storage media maybe used to provide instructions to processing unit 110 for execution.Such instructions, generally referred to as “computer program code”(which may be grouped in the form of computer programs or othergroupings), when executed, enable the infrared imaging system 100 toperform features or functions of embodiments of the current technology.Further, as used herein, “logic” may include hardware, software,firmware, or a combination of thereof.

Before describing other figures of the disclosure, an overview of colormodels and palettes utilized in providing user-viewable images (e.g., ona display device) is given. A palette typically comprises a finite setof color and/or grayscale representations selected from a color model(e.g., grayscale values in black and white or other monochrome colors,or color values in RGB, CIEXYZ, CIELab, or any other known color modelsuitable for representing colors and/or grayscales on a display device)for false coloring or pseudo-coloring of images through assigning of thecolor and/or grayscale representations to pixels according to predefinedmapping rules. A predefined palette represents a finite set of colorand/or grayscale values of a color model displayable on a display devicethereby making image data visible to the human eye.

In some embodiments, methods comprise generating a visual representationof captured infrared data values by mapping color and/or grayscalevalues to each pixel of a captured frame of infrared data values,thereby assigning each pixel of the frame of infrared data values arepresentation value from a color model. Pixel value components mayrepresent various values. For example, one type of pixel value componentrepresents colors as a combination of different color channels orchromaticity values (combined through addition and subtraction), as acombination of hue, saturation, and intensity/brightness, or as otherrepresentations appropriate for desired color models.

By way of example, an RGB color model is an additive color model anduses the combination of primary colors red, green, and blue to constructall other colors, as would be understood by one skilled in the art. Acolor-representing component in a visual representation of the capturedIR data values may thus be expressed as the combination of intensitiesof the three primary colors according to an RGB color model, in otherwords as a 3-tuple (R, G, B) representing intensities in the threeprimary colors. For example, if assuming the intensities R, G, B cantake on 256 discrete values from 0 to 255 (such as when the intensitiesare represented by 8-bit integers), 16,777,216 possible colors can beproduced by an additive combination of the 256 discrete intensity levelsof the three primary colors, with the color white being an equalcombination of all three base colors at maximum intensity (255,255,255)and the color black being zero intensity for each component (0,0,0).Although 256 discrete primary color intensity levels have beenillustrated in the example above, a 3-tuple for the color-representingcomponent may permit a different number of available intensity levelsand/or a different expression (e.g., expressed as a fraction of maximumintensity) as would be understood by one skilled in the art.

In other examples, the color-representing component for a visualrepresentation may be expressed as a combination of hue, saturation, anda brightness value according to an HSV (also referred to as HSB) colormodel, a combination of hue, saturation, and lightness according to anHSL color model, or other appropriate combinations according to a colormodel known in the art. As also understood by one skilled in the art, acolor-representing component expressed in one color model can beconverted into another color model using known formulae.

It is to be understood that although embodiments of the presentdisclosure are illustrated using an RGB color model as examples, oneskilled in the art having viewed the disclosure would be able to modifythe embodiments according to the principles taught by the embodiments toutilize other color models, such as but not limited to HSL, HSV, CMYK,NTSC, and ROMM, without departing from the scope of this invention.

Turning now to FIGS. 2A-2C, example visual representations 210A-210C ofa thermal IR image (a thermal image of a circuit board with integratedcircuits (ICs) and other components in these examples) are shown inaccordance with prior art techniques. In particular, FIGS. 2A-2Crespectively show a conventional palette 200A designed around the colorsof a rainbow (also referred to herein as a rainbow palette), aconventional grayscale palette 200B (also referred to herein as awhite-hot palette), and a conventional palette 200C designed around thecolors of incandescent objects (also referred to herein as a fusion orironbow palette), which are applied to the same 14-bit thermal IR imageto produce visual representations 210A-C. Portion 212A (e.g.,representing a large IC in a circuit board) and portion 214A (e.g.,representing small ICs in the circuit board) of infrared image 210A showlow contrast, resulting in difficult determination of temperature valuevariations.

Corresponding portions 214B-C of visual representations 210B-C appear tobe depicted as a singular temperature by the conventional colormappings. Only portions 216A-C located at the bottom of visualrepresentations 210A-C display substantial temperature value transitionsand relatively noticeable differentiation between neighboring colorvalues. Color palettes 200A-C, which are illustrated as color bars onthe right edge of visual representations 210A-C in these examples, showthe various colors corresponding to a range of output levelsrepresenting IR intensity or temperature. These colors in color palettes200A-C may be represented by color values (e.g., 3-tuples) according toan RGB color model having 256 discrete intensity levels for each primarycolor, for example.

Example plots of the conventional palettes 200A-C used for generatingvisual representations 210A-C are shown in FIGS. 3A-3C. In FIGS. 3A-3C,the RGB color values in the palettes used in generating visualrepresentations 210A-C are plotted using Cartesian coordinates and256-point trajectories in RGB space as plots 304A-C, respectively. Plots302A-C are isometric views into the RGB-space origin at (0,0,0), whichis a pure black color value. As plots 302A-C show, the inventor hasdiscovered that there is a significant amount of RGB space not utilizedby the rainbow palette, whitehot palette, or ironbow palette. Inaddition, distances between adjacent individual output levels aresubstantially small, thus making it difficult to distinguish differencesbetween nearby output levels (i.e., neighboring color values in apalette that represent different temperature values).

For example, conventional palettes such as palettes 200A-C map 256discrete output levels to corresponding 256 color values. Each of the256 color values in a palette (e.g., expressed as 3-tuples of 256discrete primary color intensity levels in RGB) is plotted at acoordinate in the three-dimensional RGB space (the primary colors of theRGB space are red, green, and blue, which are located at the corners ofthe Euclidean space). Plotting the palettes as a scatter plot in RGBspace shows: the extent to which the palette uses the available volumein an RGB space and the closeness of the individual output levels in anRGB space (if the spacing is too close, it is difficult to distinguishvariances and differences between nearby output levels).

For example, in rainbow palette 200A, the distance between color valuescorresponding to adjacent output levels is relatively small in an RGBcolor space, as plot 302A shows, thus resulting in relatively lowperceived contrast. Therefore, the rainbow palette is not a suitablecontrast optimizer; it is a familiar color scheme based on the familiarvisible light spectrum scheme,Red-Orange-Yellow-Green-Blue-Indigo-Violet. Rainbow palette plot 302Adisplays relatively saturated hues save the highest and lowesttemperatures, which, for example, may rely more on tints and shades,respectively. Grayscale palette plot 302B shown in FIG. 3B is a short,straight line. The short, straight line represents grayscale variationswhich lie on the diagonal of the color solid to cover 256 discreteoutput levels. However, the human eye can only distinguish 50-60variations of gray resulting in approximately 200 indistinguishablegreys used in the grayscale palette. Ironbow palette 200C, ascorresponding scatter plot 302C in FIG. 3C demonstrates, has similarcomplications, acting merely as part of an infrared vernacular and usingeven less space than the rainbow palette. Such shortcomings of theconventional palettes make subtle variations in temperatures across theICs in the exemplary images 210A-C a laborious read.

Various embodiments of the disclosure discussed herein below mayovercome such shortcomings of using conventional palettes to generatevisual representations of an IR image. FIG. 4 illustrates a palette 400in accordance with an embodiment of the disclosure. Palette 400 maycomprise or define a sequence of color values 402 corresponding to arange of output levels 404. In this regard, palette 400 is illustratedin FIG. 4 as a LUT indexed by a range of discrete output levels 404(e.g., an array indexed by output levels 404) for looking up (e.g.,accessing or indexing) corresponding color values 402 represented in3-tuples of an RGB color model. However, as discussed above, palette 400may be implemented in other data structures or as a functional module(e.g., to calculate or otherwise determine corresponding color values inresponse to receiving output levels) according to the principlesdiscussed herein, without departing from the scope and spirit of thepresent disclosure. Note that the column labeled “Color Name” is shownfor purposes of illustration, and not necessarily a part of palette 400.

Certain color values in the sequence of color values may be identifiedas base color values or breakpoints (collectively identified as basecolor values 406 or individually identified as base color values 406(1)through 406(6) in the example of FIG. 4), according to embodiments. Basecolor values 406 may be positioned at certain intervals in the sequenceof color values 402, for example, at substantially similar intervalspartitioning the sequence. In the example palette 400, six base colorvalues 406 are shown at 51-row intervals between successive ones topartition the 256-long sequence of color values, such that base colorvalue 406(1) corresponds to output level 1, base color 406(2)corresponds to output level 52, and so on as shown in FIG. 4. Becausethe length of the color value sequence may not always be whollydivisible by the number of intervals between base color values, theintervals between them may not always be the same but substantiallysimilar (e.g., +/−2). The color values in palette 400 identified as basecolor values 406 thus define a series of base color values 406, wheresuccessive ones of the base color values in the series are atsubstantially similar intervals in the sequence of color values 402.

According to various embodiments, the color values of the sequencebetween two successive base color values (e.g., between base color value406(1) and 406(2), between 406(2) and 406(3), between 406(3) and 406(4),between 406(4) and 406(5), and between 406(5) and 406(6)) may beinterpolated values of the two successive base color values. Forexample, in some embodiments, the intensity levels for the primarycolors R, G, B may individually change by linear interpolation betweenthe RGB intensity levels of two successive base color values. In otherembodiments, non-linear interpolation techniques or other interpolationtechniques may be used to determine the color values between twosuccessive base color values, for example in connection with colorvalues represented in other color models.

The series of base color values 406 are selected to improve theperceptual color distance between colors corresponding to adjacentoutput levels, in accordance with various techniques of the disclosurediscussed with additional reference to FIGS. 5-16 below. For example,the series of base color values 406 may comprise successive base colorvalues that represent pairs of colors that are complementary to eachother, or the series of base color values 406 may comprises two or moresubseries that each represent a different saturation and/or intensity ofa same set of predetermined hues, as further described herein.

FIG. 5 illustrates how an IR image data value of each pixel in an IRimage, such as those captured by image capture component 130 of system100 for example, is used to determine a corresponding color valueaccording to a palette, in accordance with an embodiment of thedisclosure. As discussed above with reference to FIG. 1, an IR image(e.g., IR image data for a frame of IR video or still image) maycomprise a plurality of pixels each having an IR data value representingan intensity of IR radiation received by a corresponding detectorelement of an IR imaging sensor. Such IR data values 520 may be from arange of discrete intensity values (also referred to as counts)available depending on the implementation of image capture component130. In the example illustrated by FIG. 5, IR data values 520 may takeon any one of 16,384 discrete values represented by a 14-bit integer,but in other embodiments IR data values 520 may take on a value fromother ranges of counts (e.g., 65,536 discrete values represented by a16-bit integer).

To determine a corresponding color value for IR data value 520 of apixel according to a palette 500 (such as palette 400), IR data value520 may be mapped to an output level 522 from a range of discrete outputlevels 504 associated with palette 500, in some embodiments. Outputlevel 522 mapped from IR data value 520 can be used to look up acorresponding one of color values 502 in palette 500, as described abovewith reference to FIG. 4. In the example illustrated by FIG. 5, IR datavalue 520, which may take on any one of 16,384 discrete valuesrepresented by a 14-bit integer, is mapped to output level 522 from arange of 256 discrete output levels to look up, access, or otherwisedetermine a corresponding color value according to palette 500 thatcomprises a sequence of 256 color values 502 associated with 256discrete output levels 504. As may be appreciated, in other embodiments,output level 522 may take on a value from other ranges of output levelsassociated with a particular implementation of palette 500.

In some embodiments, output level 522 may be determined from IR datavalue 520 according to a user-defined mapping, such as when a user ofsystem 100 sets via control component 150 a temperature range to bevisualized by false color or pseudo-coloring. In some embodiments,output level 522 may be determined from IR data value 520 according toautomatic ranging based on the span of all IR data values (e.g.,automatically determined mapping relative to the span of all IR datavalues) captured in a frame of IR image. For example, known histogramanalysis techniques such as those described in connection with FIG. 4 ofU.S. patent application Ser. No. 14/582,736 filed Dec. 24, 2014 andentitled “Augmented Image Generation” which is hereby incorporated byreference in its entirety, or other appropriate auto ranging techniquesmay be used for mapping IR data value 520 to output level 522.

In some embodiments, output level 522 may be determined from IR datavalue 520 according to a fixed mapping between the two. Embodiments inwhich output level 522 is determined from IR data value 520 by a fixedmapping may provide a radiometric visual representation (e.g., where aspecific color represents a specific temperature or IR intensity) of thecaptured IR image depending on the implementation of the palette, asfurther discussed herein. System 100 may be configured to allow the userto select (e.g., via control component 150) between the mappingsdiscussed above for various embodiments.

FIGS. 6A-6B show visual representations 610A and 610B generatedaccording to palettes 600A and 600B comprising two or more subseries ofbase colors to improve the perceived color distance or contrast betweenadjacent output levels, in accordance with one or more embodiments ofthe present disclosure. Palettes 600A and 600B are illustrated in FIGS.6A and 6B as color bars for ease of understanding, and may also bereferred to herein as a “Double Rainbow palette” and a “Triple Rainbowpalette,” respectively.

In some embodiments, series of base color values 606A (individuallyidentified as base color values 606A(1) through 606A(14)) in the DoubleRainbow palette (palette 600A) may comprise two subseries of base colorvalues, the first subseries 608A(1) comprising base color values 606A(1)through 606A(7) and the second subseries 608A(2) comprising base colorvalues 606A(8) through 606A(14). As shown, the second subseries 608A(2)may repeat the base color values of the first subseries 608A(2) at adifferent saturation and/or intensity (brightness or lightness) than thebase color values of the first subseries 608A(2). In this regard, boththe first and second subseries 608A(1) and 608A(2) comprise base colorvalues representing a same set of predetermined hues, but each with adifferent saturation and/or intensity (brightness or lightness). Forexample, in the Double Rainbow palette (palette 600A), the set ofpredetermined hues may comprise the hues of a rainbow (e.g., red,orange, yellow, green, blue, indigo, and violet), and the firstsubseries 608A(1) may comprise base color values 606A(1)-606A(7) thatrepresent the hues of a rainbow with a certain saturation and/orintensity (e.g., fully saturated or in a pure hue) and the secondsubseries 608A(2) may comprise base color values 606A(8)-606A(14) thatrepresent the hues of a rainbow in a different saturation and/orintensity (e.g., in a diminished saturation). Color values betweensuccessive base color values 606 are interpolated as discussed abovewith reference to FIG. 4.

Similarly, in some embodiments, series of base color values 606B(individually identified as base color values 606B(1) through 606B(21))in the Triple Rainbow palette (palette 600B) may comprise threesubseries of base color values 608B(1), 608B(2), and 608B(3), where thefirst subseries 608B(1) may comprise base color values 606B(1)-606B(7)that represent the hues of a rainbow in a certain saturation and/orintensity, the second subseries 608A(2) may comprise base color values606B(8)-606B(14) that represent the hues of a rainbow in a differentsaturation and/or intensity than the first subseries 608B(1), and thethird subseries 608A(3) may comprise base color values 606B(15)-606B(21)that represent the hues of a rainbow in a saturation and/or intensitydifferent from the first and the second subseries 608B(1) and 608B(2).

Although palettes 600A and 600B having two or three subseries of basecolor values are illustrated as examples, palettes in other embodimentsmay comprise four or more subseries of base color values to furtherincrease the perceived color distance/contrast between adjacent outputlevels. In addition, although subseries 608A and 608B of base colorvalues use the hues of a rainbow as an example, other hue combinationsmay be utilized as understood by one skilled in the art.

The use of varying saturations and/or intensities of the base colors insuch ways allows a viewer to better distinguish small color valuesvariations, and thus, temperature changes. For example, FIG. 6A showsportions 614A and 614B corresponding to an IC in visual representation610A and 610B of a circuit board that is clearly distinguishable fromthe rest of the circuit board when palettes 600A and 600B are applied,whereas, corresponding portions 214A-214C in visual representations210A-210C generated using conventional palettes 200A-200C areindistinguishable from the surrounding circuit board areas. Portions612A and 612B show substantial contrast relative to correspondingportions 212A-212C with traditional palettes 200A-200C applied. Portions612A and 612B display three hues and the corresponding interpolatedcolors. Portions 616A and 616B also display a relatively large array oftemperature differentials (e.g., up to 7 distinct hues are shown overportions 616A and up to 10 distinct hues are shown over portions 616B)in comparison to corresponding portions 216A-216C of conventionalpalette images 210A-210C.

As illustrated in FIGS. 7A and 7B, the RGB-space trajectory plots 702Aand 702B of palettes 600A and 600B respectively cover a greater portionof the RGB space than the trajectory plots 302A-302C of conventionalsingle rainbow, grayscale, or ironbow palettes. Furthermore, thesequences of color values in palettes 600A and 600B show substantialdistances between adjacent output levels. However, distances betweenadjacent output levels may still be limited due to the base colors beingbased on an arbitrary or familiar set of hues (e.g., a rainbow) and nothow the human vision perceives color contrasts.

FIG. 8A shows a close-up view 814A of portion 614A in visualrepresentation 610A, where palette 600A comprises a sequence of 256color values corresponding to a range of 256 distinct output levels(e.g., from output level 1 through 256), which is a typical size of apalette in IR imaging systems. As can be seen in FIG. 8A, steps (orbanding) in color transitions are noticeable, especially since palette600A advantageously increases the perceived color distances betweendifferent temperatures or IR intensities. Such steps or banding effectsin visual representations may be perceived as unnatural by some users.Thus, in some embodiments, the number of distinct output levels mappedby palettes 600A and/or 600B, and hence the number of color values inpalettes 600A and/or 600B, may be increase to further enhance visualrepresentation 610A and/or 610B of IR image data.

For example, FIG. 8B shows a close-up view 814B of portion 614A invisual representation 610A, where palette 600A comprises a sequence of512 color values corresponding to a range of 512 distinct output levels(e.g., from output level 1 through 512). As can be seen from FIG. 8B,the color transitions are much smoother without noticeable steps orbanding effects in embodiments with palette 600A having 512 distinctoutput levels. Other embodiments are contemplated in which the palettecomprises a sequence of 1024 or more color values corresponding to arange of 1024 or more distinct output levels. Palettes having a largersequence of color values may especially be beneficial to reducestairstep or banding effects in visual representations generatedaccording to various techniques of the present disclosure to increasecolor distances or contrasts between adjacent output levels.

According to some embodiments, base colors for a palette may be chosenbased on how the human vision perceives color contrasts, rather thanbeing based on an arbitrary or familiar set of hues (e.g., a rainbow).Thus, in some embodiments, the series of base color values in a palettecomprise successive base color values representing colors that arecomplementary to each other to increase the perceptual color distancebetween base colors, and hence the perceptual color distance between thecolors of adjacent output levels.

Referring again to FIG. 4, palette 400 may represent a palette of suchan embodiment. In the example palette 400, successive base color values406(1) and 406(2) represent the colors blue and yellow, respectively,which are complementary to each other in an RGB color model. Similarly,successive base color values 406(3) and 406(4) represent a pair ofcomplementary colors magenta and green, and successive base color values406(5) and 406(6) represent a pair of complementary colors red and cyanin an RGB color model. More specifically in this example, primary colors(blue, red, and green) in an RGB color model, and their respectivecomplementary colors, which are secondary colors (e.g., yellow, magenta,and cyan obtained by combining two primary colors) in an RGB colormodel, are represented by the three pairs of successive base colorvalues. However, other complementary color pairs as understood by oneskilled in the art may be used in other embodiments.

In addition, base color values 406 represent colors at their maximumintensities in the example palette 400. That is, if base color values406 are expressed in 3-tuples (R, G, B) as in the example palette 400,at least one element of each 3-tuple has the maximum value (e.g., thevalue of 255 if the intensities are expressed in an 8-bit integer) andcomplementary color value pairs can be determined as color value pairswhose additive combination of the 3-tuples has the maximum value for allcomponents (e.g., (255, 255, 255)). Thus, for example, successive basecolor value pairs 406(1) and 406(2) respectively have a color valueexpressed in a 3-tuple (0, 0, 255) for blue and a 3-tuple (255, 255, 0)for yellow, whose additive combination results in the 3-tuple (255, 255,255). Base color values that represent primary and secondary colors attheir maximum intensities in an RGB color model lie at corners of an RGBcolor space, and thus can beneficially create larger spacing betweenadjacent output levels, as can be visualized in FIG. 9 below. It shouldbe understood that in other embodiments based on color models other thanan RGB color model, base color values representing primary and secondarycolors at their at their maximum intensities, saturation, and/orbrightness according to such color models may be used to achieve thesame or similar effect. Furthermore, other complementary color pairs atless than the maximum intensities may be used in other embodiments asdesired for particular applications.

The odd-even successive base color value pairs 406(1) and 406(2), 406(3)and 406(4), and 406(5) and 406(6) represent complementary color pairs asdiscussed above. In addition, the even-odd successive base color valuepairs 406(2) and 406(3), and 406(4) and 406(5) represent pairs of colorsthat are near-complementary to each other in the example palette 400. Inother words, the sequence of base color values 406 is ordered such thatsuccessive base color values, even when they are not representing acomplementary color pair, are as far apart as possible in an RGB colorspace. This may be achieved, for example, by ordering the sequence ofbase color values 406 such that an additive combination of anon-complementary successive base color value pair results in a 3-tuplehaving two elements at their maximum value (e.g., (510, 255, 255), (255,255, 0)). However, other arrangement or ordering of non-complementarysuccessive base color value pairs may be used for other embodiments asdesired.

FIG. 9 shows a trajectory plot 902 of the example palette 400 in an RGBspace, in accordance with an embodiment of the disclosure. Base colorvalues 406 are identified in plot 902, which demonstrates thatsuccessive base color values are at farthest corners (for complementarycolors) or at second farthest corners (for near-complementary colors) ofeach other in the RGB space to create larger spacing between adjacentoutput levels. Moreover, RGB-space trajectory plot 902 covers a greaterportion of the RGB space than trajectory plots 302A-302C of conventionalsingle rainbow, grayscale, or ironbow palettes, and even the trajectoryplots 702A and 702B of the Double and Triple Rainbow palettes.

An example visual representation 1010 generated according to palette 400is shown in FIG. 10, in accordance with an embodiment of the disclosure.Palette 400 is illustrated as a color bar in FIG. 10 for ease ofunderstanding. As can be seen from FIG. 10, portions 1012, 1014, and1016 (e.g., representing various ICs in a circuit board) are much moredistinguishable and the temperature differentials created by circuittraces on the circuit board are much more discernible in visualrepresentation 1010, as compared with visual representations 210A-210Cgenerated using conventional palettes 200A-200C.

Another embodiment in which the series of base color values in a palettecomprises successive base color values representing colors that arecomplementary to each other is illustrated with reference to FIG. 11.FIG. 11 illustrates base color values 1106 (individually identified as1106(1) through 1106(12)) of a palette 1100 arranged as a color wheelfor a more intuitive understanding, in accordance with an embodiment ofthe disclosure. Arranged as a color wheel, complementary colors areplaced opposite to each other, with their base color values in 8-bit RGB3-tuples and the base color value sequence also shown.

As shown, the embodiment illustrated by FIG. 11 utilizes morecomplementary color pairs than the example palette 400. In particular,six pairs of complementary colors are represented by base color values1106, including three primary-secondary color complementary pairs as inthe example palette 400, as well as three tertiary-tertiary color (i.e.,a color obtained by a combination of a primary and a secondary color inequal portions) complementary pairs. As with the example palette 400,the color represented by base color values 1106 are at their maximumintensities in an RGB color model. That is, base color values 1106 haveat least one element of each 3-tuple at the maximum value (e.g., thevalue of 255 if the intensities are expressed in an 8-bit integer), andthus lie at corners or on edges of an RGB color space to create largerspacing between adjacent output levels. Also, non-complementarysuccessive base color value pairs (e.g., successive base color value1106(2) and 1106(3), and other even-odd successive base color valuepairs) represent colors that are near-complementary to each other,similar to the example palette 400. Note that the tertiary colors attheir maximum intensity according to the example above may have thevalue 127 or 128 (depending on how fractions are rounded) as one elementof the 8-bit RGB 3-tuple base color value, and thus an additivecombination of a tertiary-tertiary color complementary base color valuepair may result in one element of the 8-bit RGB 3-tuple having a valueclose to, but not exactly, the maximum value of 255 due to roundingerrors.

FIG. 12 shows a trajectory plot 1202 in an RGB space of the examplepalette 1100 illustrated by FIG. 11, in accordance with an embodiment ofthe disclosure. Base color values 1106 are identified in plot 1202,which demonstrates the even larger spacing created by using twelvesuccessive base color values that are complementary ornear-complementary to each other when compared with trajectory plot 902of the example palette 400 having six successive base color values 406that are complementary or near complementary to each other. Moreover,RGB-space trajectory plot 1202 covers an even greater portion of the RGBspace than trajectory plots 902 of palette 400 having six successivebase color values 406.

An example visual representation 1310 generated by applying palette 1100illustrated by FIG. 11 is shown in FIG. 13, in accordance with anembodiment of the disclosure. Palette 1100 is illustrated as a color barin FIG. 13 for ease of understanding. As can be seen from FIG. 13, thetemperature differentials in portions 1312, 1314, and 1316 (e.g.,representing various ICs in a circuit board) and the temperaturedifferentials created by circuit traces and other components on thecircuit board are much more discernible in visual representation 1310 ascompared with visual representation 1010 generated using palette 400.

The example palettes 400 and 1100 according to various embodiments ofthe disclosure do not repeat or reuse base color values. Thus, if IRdata values 520 of a captured IR image determine output levels 522according to a fixed mapping as discussed above in connection with FIG.5, visual representations 1010 and 1310 generated by applying theexample palettes 400 and 1100 will be radiometric images having aone-to-one correspondence between colors and temperatures (or IRintensity levels) observed from a scene.

A palette 1400 that comprises successive base color values 1406representing pairs of colors that are complementary to each other isillustrated in FIG. 14, in accordance with yet another embodiment of thedisclosure. Base color values 1406 may also be individually identifiedas base color values 1406(1) through 1406(26) corresponding to the basecolor number shown in FIG. 14. The sequence of color values in palette1400 other than base color values 1406 are omitted in FIG. 14 forclarity. As discussed herein, such other color values in the sequenceare interpolated between successive base color values.

In palette 1400, complementary color pairs are reused or repeated in thesequence of base color values 1406 to provide 26 base color values1406(1) through 1406(26). More specifically, for example, base colorvalues 1406(1) through 1406(12) (also identified as subseries 1409(1) inFIG. 14) represent the six pairs of complementary colors identified withrespect to palette 1100 above, whereas base color values 1406(13)through 1406(24) (also identified as subseries 1409(2) in FIG. 14) alsorepresent the six pairs of complementary colors but in different ordersthan subseries 1409(1). In other embodiments, the six pairs ofcomplementary colors may be repeated in the same order in the sequenceof base color values 1406. Successive base color values 1406(25) and1406(26) repeat the complimentary color pair blue and yellow to pad theend of the sequence of color values in palette 1400. Also in thisexample, every pair of successive base color values 1406 in palette 1400represent a pair of colors that are complementary or near-complementaryto each other.

In other embodiments, not all six pairs of complementary colors need tobe repeated, but instead one or more pairs selected from the six pairsmay be repeated. For example, a palette comprising successive base colorvalue pairs representing seven pairs (the six pairs plus one repeatedfrom the six pairs), eight pairs (the six pairs plus two repeatedpairs), nine pairs (the six pairs plus three repeated pairs), ten pairs(the six pairs plus four repeated pairs), or eleven pairs (the six pairsplus five repeated pairs) may be provide depending on embodiments. Inyet other embodiments, two or more pairs from the six pairs ofcomplementary colors may be repeated more than twice in the sequence ofbase color values 1406.

FIG. 15 shows a trajectory plot 1502 of the example palette 1400 in anRGB space, in accordance with an embodiment of the disclosure. Thespacing (or the perceptual color distance) between two adjacent outputlevels are even greater in plot 1502 than in trajectory plot 1202 of theexample palette 1100 not having the repeated pairs. Thus, applying ahigh-base-color-count palette such as palette 1400 may beneficiallycreate large local contrast in a visual representation of IR image data.

Such a beneficial effect may be observed in an example visualrepresentation 1610 generate by applying palette 1400, as shown in FIG.16 in accordance with an embodiment of the disclosure. In visualrepresentation 1610, local temperature differentials, for example inportions 1612, 1614, and 1616, are exaggerated by high color contrast,which is useful for discerning and interpreting small temperaturedifferentials in localized areas in a scene. However, visualrepresentation 1610 may not be useful for global temperaturemeasurement, since it is not radiometric due to the repetition of basecolors in palette 1400.

Turning now to FIG. 17, a flowchart is shown of a method 1700 forcapturing and processing infrared images in accordance with anembodiment of the present disclosure. For purposes of simplifyingdiscussion of FIG. 17, reference may be made to imaging system 100 ofFIG. 1 as an example of a system, device, or apparatus that may performmethod 1700. However, method 1700 may be performed by other suitableimaging systems, devices, or apparatuses as would be understood by oneskilled in the art.

At block 1710, one or more images (e.g., IR image data or signal) may becaptured using infrared imaging system 100. In one embodiment,processing component 110 induces (e.g., causes) image capture component130 to capture an image, such as a thermal infrared image of scene 170.For example, IR image data captured by an IR imaging sensor of imagecapture component 130 may comprise a plurality of pixels each having anIR data value representing an intensity of IR radiation received bycorresponding detector elements of the IR imaging sensor. Afterreceiving the captured image (e.g., IR image data) from image capturecomponent 130, processing component 110 may optionally store thecaptured image in memory component 120 for processing.

Next, the captured image may optionally be pre-processed at block 1720.In one embodiment, pre-processing may include obtaining infrared sensordata related to the captured image, applying correction terms, and/orapplying temporal noise reduction to improve image quality prior tofurther processing. In one embodiment, processing component 110 maydirectly pre-process the captured image or optionally retrieve thecaptured image stored in memory component 120 and then pre-process theimage. Pre-processed images may be optionally stored in memory component120 for further processing.

At block 1730, a palette may be selected to use in determining colorvalues that corresponds to the captured IR data values. For example,such a palette may be selected from one or more palettes that improvethe perceptual color distance between adjacent output levels inaccordance with various techniques discussed above, such as palettes400, 600A, 600B, 1100, and/or 1400 (e.g., stored as palettes 112 inmemory component 120). In various embodiments, the palette to be used indetermining color values for the captured IR data values may be selectedin response to receiving a user input via control component 150 of IRimaging system 100 for example, or automatically selected by processingcomponent 110, or both. In this regard, according to some embodiments, auser of IR imaging system 100 may be presented with a menu one ofseveral scenarios, applications, or modes (identified in the menu bynames indicative of the scenarios, e.g., man overboard, maintenance,electrical systems surveillance or inspection, heating or ventilation,or other situations where IR imaging may be useful) on a display orcontrol panel, so that the user may select an applicable use scenario.In such embodiments, processing component 110 may be configured toselect an appropriate one of the available palettes based on the user'sselection of the use scenario, and optionally further based on ananalysis of the captured IR image data (e.g., based on the analysis ofhow much local contrast is present in the captured IR image).

At block 1740, a visual representation of at least a portion of thecaptured IR image data may be generated according to the varioustechniques discussed in detail above with reference to FIGS. 4 through16. Thus, for example, an output level may be determined based on the IRdata value for each pixel of the at least a portion of the captured IRimage data, and then a color value corresponding to the determinedoutput level may be determined according to the selected palette, wherethe selected palette is provided according to the various techniques ofthe present disclosure to improve the perceived color distance betweenadjacent output levels. In one embodiment, processing component 110 maystore the generated visual representation in memory component 120 fordisplaying.

At block 1750, the generated visual representation may be displayed. Forexample, in one embodiment, processing component 110 may retrieve thegenerated visual representation stored in memory component 120 anddisplay the visual representation on display component 140 for viewingby a user. Display component 140 and/or control component 150 may alsobe configured to allow for interaction (e.g., editing) with processedimage via touch screen, interface, or other known methods of interactionwith an electronic device.

In some embodiments, the displaying of the visual representation atblock 1750 may include converting the generated visual representation inone color model into a representation in another color model used bydisplay component 140. In some embodiments, the displaying of the visualrepresentation at block 1750 may include marking (e.g., colorizing,highlighting, or identifying with other indicia) certain features in thevisual representation, for example, to aid a user to identify thesefeatures while viewing the displayed image. In some embodiments, thedisplaying of the visual representation at block 1750 may includeoverlaying or appending a representation (e.g., a color bar with orwithout a corresponding temperature range) of the palette being appliedto generate the visual representation. In some embodiments, thedisplaying of the visual representation at block 1750 may includefusing, overlaying, or otherwise combining the visual representation ofan IR image with a non-thermal image (e.g., a visible light imagecaptured by visible light imaging device 190) to enhance the visualrepresentation of the captured IR image data with further contrastand/or detail extracted from the non-thermal image according to knowntechniques, such as for example those described in U.S. Pat. No.8,520,970 issued Aug. 27, 2013 and entitled “Infrared Resolution andContrast Enhancement with Fusion” and in U.S. Pat. No. 8,565,547 issuedOct. 22, 2013 and entitled “High Contrast Fusion,” which areincorporated herein by reference by their entireties.

As discussed herein above, various embodiments of the present disclosurecan provide enhanced visual representations of IR images by utilizingpalettes that advantageously improve the perceived color distance orcontrast between colors corresponding to adjacent output levels (e.g.,similar temperature or IR intensity) in IR images. Thus, for example,users of an infrared imaging system according to various embodiments ofthe disclosure may be provided with enhanced visual representations ofIR images that beneficially allow users to discern subtle localtemperature or IR intensity differences for a better understanding andinterpretation of the scene or object that the users are inspecting orotherwise monitoring.

Where applicable, various embodiments of the invention may beimplemented using hardware, software, or various combinations ofhardware and software. Where applicable, various hardware componentsand/or software components set forth herein may be combined intocomposite components comprising software, hardware, and/or both withoutdeparting from the scope and functionality of the invention. Whereapplicable, various hardware components and/or software components setforth herein may be separated into subcomponents having software,hardware, and/or both without departing from the scope and functionalityof the invention. Where applicable, it is contemplated that softwarecomponents may be implemented as hardware components and vice-versa.

Software, in accordance with the invention, such as program code and/ordata, may be stored on one or more computer readable mediums. It is alsocontemplated that software identified herein may be implemented usingone or more general purpose or specific purpose computers and/orcomputer systems, networked and/or otherwise. Where applicable, orderingof various steps described herein may be changed, combined intocomposite steps, and/or separated into sub-steps to provide featuresdescribed herein.

While the invention has been described in detail in connection with onlya limited number of embodiments, it should be readily understood thatthe invention is not limited to such disclosed embodiments. Rather, theinvention can be modified to incorporate any number of variations,alterations, substitutions or equivalent arrangements not heretoforedescribed, but which are commensurate with the spirit and scope of theinvention. Additionally, while various embodiments of the invention havebeen described, it is to be understood that aspects of the invention mayinclude only some of the described embodiments. Accordingly, theinvention is not to be seen as limited by the foregoing description, butis only limited by the scope of the appended claims.

What is claimed is:
 1. A method comprising: receiving infrared (IR)image data captured by an IR imaging sensor comprising a plurality ofdetector elements, wherein the IR image data comprises, for eachdetector element, a corresponding pixel having an IR data valuerepresenting an intensity of IR radiation received by the detectorelement; and generating a visual representation of at least a portion ofthe IR image data, wherein the visual representation comprises, for eachpixel of the at least the portion of the IR image data, a correspondingcolor-representing component according to a color model, and wherein thegenerating the visual representation comprises, for each pixel:determining an output level based on the IR data value of the pixel,determining a color value that corresponds to the determined outputlevel according to a palette, and assigning the determined color valueto the corresponding color-representing component of the visualrepresentation; wherein the palette comprises a sequence of color valuescorresponding to a range of discrete output levels, the sequence ofcolor values comprising a series of base color values positioned atsubstantially similar intervals partitioning the sequence, wherein colorvalues in the sequence other than the base color values are interpolatedbetween successive base color values; and wherein the series of basecolor values comprises: at least one pair of successive base colorvalues representing a pair of colors that are complementary to eachother, or two or more subseries each representing a different saturationand/or intensity of a same set of predetermined hues.
 2. The method ofclaim 1, wherein the series of base color values comprises at leastthree pairs of successive base color values, each pair representing apair of colors that are complementary to each other; and wherein colorsrepresented by the at least three pairs of successive base color valuescomprise primary and secondary colors at their maximum intensities,saturation, and/or brightness according to the color model.
 3. Themethod of claim 1, wherein the series of base color values comprises atleast six pairs of successive base color values, each pair representinga pair of colors that are complementary to each other; and whereincolors represented by the at least six pairs of successive base colorvalues comprise primary, secondary, and tertiary colors at their maximumintensities, saturation, and/or brightness according to the color model;or wherein the series of base color values further comprises at leastone pair of successive base color values that are repeated from the atleast six pairs of successive base color values representingcomplementary colors; or wherein the series of base color valuescomprises two or more subseries that each comprise, in different or sameorders, the at least six pairs of successive base color valuesrepresenting complementary colors.
 4. The method of claim 1, wherein thesequence of base color values comprises a subsequence in which everyodd-even successive base color value pair represents a pair of colorsthat are complementary to each other and every even-odd successive basecolor value pair represents a pair of colors that are near-complementaryto each other.
 5. The method of claim 1, wherein: the range of discreteoutput levels comprises at least 1024 output levels represented in10-bit integers; and the sequence of color values comprises at least1024 color values corresponding to the 1024 output levels.
 6. The methodof claim 1, wherein: the series of base color values comprises the twoor more subseries, a first subseries of the two or more subseriescomprises base color values representing hues of red, orange, yellow,green, blue, indigo, and violet; a second subseries of the two or moresubseries comprises base color values representing the hues of red,orange, yellow, green, blue, indigo, and violet having a differentsaturation and/or intensity than the first subseries; and a thirdsubseries of the two or more subseries comprises base color valuesrepresenting the hues of red, orange, yellow, green, blue, indigo, andviolet having a different saturation and/or intensity than the first andthe second subseries.
 7. The method of claim 1, wherein the output levelfor each pixel is determined based on the IR data value of the pixelrelative to the span of all IR data values captured in the at least theportion of the IR image data; or wherein the output level for each pixelis determined by a fixed mapping between the IR data value and theoutput level of the pixel.
 8. The method of claim 1, wherein: the colormodel is an RGB color model or a hue-saturation-value (HSV) model; thecolor-representing components of the visual representation each comprisea 3-tuple representing intensities in three primary colors or a 3-tuplerepresenting hue, saturation, and brightness; each of the base colorvalues has at least one of the three primary colors at a maximumintensity; the range of discrete output levels comprises at least 512output levels represented in 9-bit integers; and the sequence of colorvalues comprises at least 512 color values corresponding to the 512output levels.
 9. The method of claim 1, further comprising: receiving acontrol signal indicative of a type of use of the visual representation;and selecting the palette from a set of palettes in response to thecontrol signal.
 10. The method of claim 1, wherein the two or moresubseries comprise two or more disjoint subseries, and wherein each ofthe two or more disjoint subseries is associated with a respectiveplurality of the base color values.
 11. The method of claim 1, whereineach discrete output level corresponds to a respective IR intensitylevel or temperature level, and wherein the series of base color valuespartitions the sequence such that, for each base color value, the basecolor value is associated with one of the discrete output levels and itsadjacent base color value is associated with an other of the discreteoutput levels, and wherein each color value in the sequence between thebase color value and its adjacent base color value is associated with adiscrete output level between the one of the discrete output levels andthe other of the discrete output levels.
 12. The method of claim 1,wherein each of the base color values is associated with a 3-tuplecomprising a first component, a second component, and a third component,and wherein at least two of the base color values have the same valuefor the first component, the same value for the second component, andthe same value for the third component.
 13. An imaging systemcomprising: an infrared (IR) imaging sensor comprising a plurality ofdetector elements, the IR imaging sensor being configured to capture IRimage data comprising, for each detector element, a corresponding pixelhaving an IR data value that represents an intensity of IR radiationreceived by the detector element; a memory configured to store a palettecomprising a sequence of color values corresponding to a range ofdiscrete output levels; a processor communicatively coupled to the IRimaging sensor and the memory, the processor being configured togenerate a visual representation of at least a portion of the IR imagedata; wherein the visual representation comprises, for each pixel of theat least the portion of the IR image data, a correspondingcolor-representing component according to a color model; wherein theprocessor is configured to generate the visual representation at leastby: determining, for each pixel of the at least the portion of the IRimage data, an output level based on the IR data value of the pixel,determining a color value that corresponds to the determined outputlevel according to the palette, and assigning the determined color valueto the corresponding color-representing component of the visualrepresentation; wherein the sequence of color values comprises a seriesof base color values positioned at substantially similar intervalspartitioning the sequence; wherein color values in the sequence otherthan the base color values are interpolated between successive basecolor values; and wherein the series of base color values comprises: atleast one pair of successive base color values representing a pair ofcolors that are complementary to each other, or two or more subserieseach representing a different saturation and/or intensity of a same setof predetermined hues.
 14. The system of claim 13, wherein the series ofbase color values comprises at least three pairs of successive basecolor values, each pair representing a pair of colors that arecomplementary to each other; and wherein colors represented by the atleast three pairs of successive base color values comprise primary andsecondary colors at their maximum intensities, saturation, and/orbrightness according to the color model.
 15. The system of claim 13,wherein the series of base color values comprises at least six pairs ofsuccessive base color values, each pair representing a pair of colorsthat are complementary to each other; and wherein colors represented bythe at least six pairs of successive base color values comprise primary,secondary, and tertiary colors at their maximum intensities, saturation,and/or brightness according to the color model; or wherein the series ofbase color values further comprises at least one pair of successive basecolor values that are repeated from the at least six pairs of successivebase color values representing complementary colors; or wherein theseries of base color values comprises two or more subseries that eachcomprise, in different or same orders, the at least six pairs ofsuccessive base color values representing complementary colors.
 16. Thesystem of claim 13, wherein the sequence of color values comprises asubsequence in which every odd-even successive base color value pairrepresents a pair of colors that are complementary to each other andevery even-odd successive base color value pair represents a pair ofcolors that are near-complementary to each other.
 17. The system ofclaim 13, wherein: the series of base color values comprises the two ormore subseries, a first subseries of the two or more subseries comprisesbase color values representing hues of red, orange, yellow, green, blue,indigo, and violet; a second subseries of the two or more subseriescomprises base color values representing the hues of red, orange,yellow, green, blue, indigo, and violet having a different saturationand/or intensity than the first subseries; and a third subseries of thetwo or more subseries comprises base color values representing the huesof red, orange, yellow, green, blue, indigo, and violet having adifferent, saturation and/or intensity than the first and the secondsubseries.
 18. The system of claim 13, wherein the output level for eachpixel is determined based on the IR data value of the pixel relative tothe span of all IR data values captured in the at least the portion ofthe IR image data; or wherein the output level for each pixel isdetermined by a fixed mapping between the IR data value and the outputlevel of the pixel.
 19. The system of claim 13, wherein: the color modelis an RGB color model or a hue-saturation-value (HSV) model; and thecolor-representing components of the visual representation each comprisea 3-tuple representing intensities in three primary colors or a 3-tuplerepresenting hue, saturation, and brightness.
 20. The system of claim13, further comprising a display device, wherein the processor isfurther configured to convert the color-representing components intorepresentations in another color model used by the display device.